Unit 4: Software Development
Activity 4: Testing
Learning Goal: By the end of this activity I will be able to design algorithms to detect, intercept and handle run-time errors and create test plans to test programs
Content
Program Correctness and Robustness
Review: Types of errors
Syntax Errors- Errors that violate the rules of Java
- Ex: forgetting a semicolon
- Found by the compiler
- An application with syntax errors will not run because it will not compile
- More difficult to detect
- Syntax is correct, but produce undesired or unexpected results
- Logic errors must be found by the programming through testing and by carefully examining source code
- Help yourself find, and prevent logic errors by:
- Carefully examining your source code
- Accurate and careful commenting
- Proper indentation
- Descriptive identifiers
- Errors not detected by the compiler
- Halts program execution at the problem statement
- Ex: division by 0 will not create a syntax error, but will generate an ArithmeticException exception
A program is correct if it performs the task that it was intended to accomplish.
A program is robust if it can handle, within reason, unexpected situations and illegal inputs by the user.
Example
You have designed a program that has the ability to read some numbers from the user and then print the same numbers in sorted order.
This program is correct if it works for any set of input numbers. Every program should be correct. A sorting program is pretty useless if it does not sort the numbers correctly.
This program is considered to be robust if it can also deal with all kinds of user input in all situations. Error-handling refers to how the program handles errors and is an important part of a robust program. One possible way that this program could handle an input error is by printing an error message and ignoring the bad input. A non-robust program may simply generate strange output or it may crash.
It's not the case that every program needs to be completely robust. It depends on who will use it and how it will be used. For example, a small utility program that you write for your own use doesn't have to be particularly robust. However, when computers are given important tasks and there is a software error, the consequences can be very serious.
Both the Console and Dialog class in the simpleIO package already perform some error-handling for you. The readInt and readDouble methods will continue to prompt the user until they enter a number of the correct type. Next unit you will build on this error-handling to create more robust computer programs.
Horror Stories
In September 1999, the Mars Climate Orbiter burned up in the Martian atmosphere because data that was expressed in English units of measurement (such as feet and pounds) was entered into a computer program that was designed to use metric units (such as centimetres and grams).
A few months later, the Mars Polar Lander probably crashed because its software turned off its landing engines too soon. The program was supposed to detect the bump when the spacecraft landed and turn off the engines then. It has been determined that deployment of the landing gear might have jarred the spacecraft enough to activate the program, causing it to turn off the engines when the spacecraft was still in the air. The unpowered spacecraft would then have fallen to the Martian surface. A more robust system would have checked the altitude before turning off the engines!
Resources
Validating User Input: The Console Class
Input Validation checks whether or not the input provided by a user meets a set of criteria. Input validation may check if the data is the correct type (text, number), within a certain range (greater than, less than, between), following a particular format (postal codes, product code), and/or is a particular length (SIN number, phone number).
The design of the Console class (that you have been using for input and output for this course) validates the user input for the type of data entered. If a user is expected to input a number, the Console class will verify that the input is actually a number before returning a value to the main method. Let's take a closer look at how this works.
Below is the code for the Console.readInt(String prompt) method that prompts the user for a number, and continues to prompt them until they enter a number that is recognized as an integer.
Line 15 contains a critical piece of code - this sets up the BufferedReader object stored in the variable br. This object is responsible for getting the text that the user types into the Console. It contains an InputStreamReader linked to the InputStream described by the data field System.in.
 The readInt() method of the Console class |
The first line of code in the readInt() method displays the prompt (given as an argument to the method) using System.out.println(). This is a PrintStream data field provided by the System class, which contains the method println(). This method outputs text to the screen (in our case, the Eclipse console). Note that all of the Console.print() methods also use this statement. After the user has been prompted, the method enters a while loop, which continues until the boolean variable validInput is set to true. The next two lines of code are contained inside a try-catch block, which is a structure designed to "catch" specific run-time errors that may happen during the execution of particular lines of code. This try-catch block is designed to catch a NumberFormatException, which would be thrown by the parseInt() method of the Integer class, in the event that the String parameter does not represent a number. It can also catch an IOException, which happens when something goes wrong with the BufferedReader object. If an error is generated, the user is given a message to help them understand why they are being prompted again (Line 75). Line 70 contains the code that actually reads the information from the user - this is done with the method call br.readLine(). The result of this is then passed to another method, Integer.parseInt(). If this line of code executes successfully (it doesn't generate a run-time error), then the validInput variable is set to true and the loop will end. After the loop is finished, the number typed in by the user is return to the application using the method. |
The other input methods of the Console class work in a similar way.
You are welcome to continue using the Console class in your programs for this course and for other courses with me. It assists with good user interface design, since it requires you to provide a user prompt when you obtain a value from the user. This way the user knows what they are expected to type. It also partially validates user input, since it will verify that the user entered a number (double or int) when required. You are welcome to modify it (or extend it) to suit your needs; the source code is provided and you can read it at anytime.
In the assignment portion of this activity, you will be creating methods that provide range validation. You should use these methods in your assessment for this unit, to prevent invalid input from crashing your program. You may find that you want to add more input validation methods as you learn more about Java and objects, or as your programs require other forms of input validation.
Test Plans
While you have had experience creating automated tests to verify that a method works, it is still important to run and test the program as whole. To ensure you are testing your programs thoroughly, you should first create a test plan.
Test plans outline all of the tests performed by the programmer, as well as the outcomes of those tests. Often the programmer will list a whole range of different potential inputs. They then list the expected outcome (which often must be determined by hand!), or what they expect to occur after they execute their program using those inputs. Finally, they run the program, enter the various inputs and record the outcomes. If some of the outcomes are different than they expected, they indicate the changes that need to be made, and then make those changes.
The programmer usually selects some regular inputs, as well as inputs that they think might cause potential problems in the program. If the problems do occur, then they list the error and then do their best to fix it.
Use a table to organize your test plan, with the headings shown below. This is a test plan for a basketball program that prompts the user for the amount of points, assists and rebounds that the user obtained. The program then determines if the user obtained a double-double or a triple-double
| INPUT | EXPECTED OUTPUT | ACTUAL OUTPUT | PASS/FAIL | PROBLEM DESCRIPTION | FIXED? |
| 5 points 2 rebounds 3 assists |
"Sorry, you did not obtain a double-double or a triple-double." | "Sorry, you did not obtain a double-double or a triple-double." | PASS | ||
| 10 points 11 rebounds 1 assists |
"You obtained a double-double." | "You obtained a double-double." | PASS | ||
| 10 points 9 rebounds 15 assists |
"You obtained a double-double." | "You obtained a double-double." | PASS | ||
| 0 points 0 rebounds 0 assists |
"Sorry, you did not obtain a double-double or a triple-double." | "Sorry, you did not obtain a double-double or a triple-double." | PASS | ||
| -2 points 11 rebounds 12 assists |
"One of your statistics is impossible to obtain. Please re-enter statistics." | "You obtained a double-double." | FAIL | The program needs to test the input data and make sure it makes sense. Input validation needs to be added to catch any negative statistics from being entered. | YES |
In the sample test plan, most of the input did not cause any problems; however, the programmer did encounter one bug, identified in the table above. If a user enters a negative number for their points, the program still runs and generates an output based on the other two statistics. The programmer wasn't expecting this; an error message should appear. Input validation is needed to fix the bug.
Recall: When creating your tests, consider the following suggestions:
- Test typical values (values you expect the user will enter)
- Test very large, very small, and both negative and positive values.
- Test boundary conditions using the extremes of your input domain (maximum, minimum, just inside or outside the boundary)
- Test all of the branches of your decision structures
- Test repetition structures for typical and large numbers of iterations, as well as zero iterations
- Test conditions that should generate an error.
Evidence of Learning
Test Plans & Java Programs
Once you have read the material above complete the following, using the evidence cards in Hapara.
- Recall from Activities 2 & 3, you were asked to create an algorithm (expressed as a flowchart or pseudocode) to solve different problems.
For this activity, you will choose TWO problems and create a test plan for the program that you will write later to solve the problem.
You must choose from the problems that you have designed in the previous activity.
You will use the table format found on the evidence card in Hapara. Consider using the examples provided as your first test cases. - The Environmental Lapse Rate (ELR) states that as you increase your altitude by 100 m, the temperature decreases by 0.65°C. Predict the temperature at any altitude, given the temperature at sea level. Altitude must be a positive number.
Ex 1: If the temperature at sea level is -5°C, at an altitude of 500m it will be -8.25°C.
Ex 2: If the temperature at sea level is 15.5°C, at an altitude of 950m it will be 9.325°C. - Given a fuel economy rating, determine the amount of fuel required for a trip. Both fuel economy and trip length must be positive numbers.
Ex 1: For a rating of 12L / 100km, a 500km trip will use 60L
Ex 2: For a rating of 7.5L / 100km, a 350km trip will use 26.25L - Given a date formatted as DD/MM/YYYY, convert into long form. Invalid dates should return error messages.
Ex 1: 05/11/1980 → November 5, 1980
Ex 2: 14/03/1995 → March 14, 1995
Ex 3: 32/13/2000 → Invalid date - Translate a word into pig latin. Your algorithm should account for words beginning with a vowel, and blends (sh, th, etc). Text that begins with a number or non-alphabetic character wuold generate an error message.
Ex 1: computer → omputercay
Ex 2: algorithm → algorithmyay
Ex 3: programming → ogrammingpray - In the Unit 4 GitHub repository (link in Hapara), there is a partially implemented ValidInput class in the activity4 package that contains six validation methods (readIntLessThan(), readIntGreaterThan(), readIntBetween(), readDoubleLessThan(), readDoubleGreaterThan(), and readDoubleBetween()).
Each method should continue to prompt the user, in a loop, until they enter a value with the specified restrictions. Click HERE for a quick example of how one method could look!
Use the ValidInputTester class to verify that your methods are working as required.
- Also in your repository, there is a Payroll application. However, it is riddled with errors! Your task is to:
- Identify the error(s) in the code files provided. You must give the type of error (syntax, logic or run-time), and a brief description of why this code will not work. Put this information in a comment above the line of code that contains the error
- Fix each error. Describe how you fixed it in the comment you created in step 1
- Run the program and check for more errors. Be especially careful of logic errors!
- Check GitHub Actions to see if you fixed most of the logic errors in the Payroll application.
Your test plans must include both valid (lots!) and invalid (just a few!) data. For this submission, you only need to complete the FIRST TWO COLUMNS (INPUT and EXPECTED OUTPUT). You may need to do some figuring on paper or with a calculator first!
You must include at least SEVEN (7) sample inputs with their expected outputs PER problem. Make sure you test a good variety of data.
Submit this document for teacher feedback BEFORE you use it for your assignment!
Commit your progress to GitHub every day. Submit the link on the Hapara evidence card when you are finished.